Papers with contrastive training
RAG-Studio: Towards In-Domain Adaptation of Retrieval Augmented Generation Through Self-Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing RAG systems that use pre-trained LLMs and retrievers often fail in specialized domains and applications. |
| Approach: | They propose a self-aligned training framework that adapts general RAG models to specific domains solely through synthetic data. |
| Outcome: | Experiments on specialized domain corpus, general LLM, and general retriever show that the self-aligned training framework outperforms human-annotated training data in specialized fields. |
The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any language (2024.naacl-long)
Copied to clipboard
| Challenge: | a recent study shows that multilingual speech processing systems can generalize to unseen languages without adaptation. |
| Approach: | They propose a phoneme-based phoneme embedding model that can be generalized to unseen languages by using a neural forced aligner. |
| Outcome: | The proposed model can generalize to unseen languages without adaptation. |
CONSCENDI: A Contrastive and Scenario-Guided Distillation Approach to Guardrail Models for Virtual Assistants (2024.naacl-long)
Copied to clipboard
| Challenge: | A major challenge in deploying LLM-based virtual conversational assistants in real world settings is ensuring they operate within what is admissible for the task. |
| Approach: | They propose to use large language models (LLMs) to generate training data with two key LLM components: scenario-augmented generation and contrastive training examples. |
| Outcome: | The proposed model improves over baselines in multiple dialogue domains. |
PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel in zero-shot document ranking tasks. |
| Approach: | They propose a prompt-based re-ranking method that requires no further training but is only feasible for reranking a handful of candidates due to computational costs. |
| Outcome: | The proposed method can retrieve documents from the entire corpus without training and with a large amount of paired text data. |
Re3Dial: Retrieve, Reorganize and Rescale Conversations for Long-Turn Open-Domain Dialogue Pre-training (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-training models lack long-turn dialogue sessions due to the scarcity of long-term sessions. |
| Approach: | They propose a framework that can automatically construct billion-scale long-turn dialogues by reorganizing existing short-turn ones. |
| Outcome: | The proposed framework can automatically construct billion-scale long-turn dialogues by reorganizing existing short-turn ones. |
TACLR: A Scalable and Efficient Retrieval-based Method for Industrial Product Attribute Value Identification (2025.acl-long)
Copied to clipboard
Yindu Su, Huike Zou, Lin Sun, Ting Zhang, Haiyang Yang, Chen Li Yu, David Lo, Qingheng Zhang, Shuguang Han, Jufeng Chen
| Challenge: | Existing methods for product attribute value identification face critical challenges . seller-provided attribute values are often incomplete or inaccurate . |
| Approach: | They propose a retrieval-based method that uses taxonomy-aware contrastive learning . they use product profiles and candidate values to encode and retrieve attributes based on similarity . |
| Outcome: | The proposed method is based on a taxonomy-aware, hard negative sampling and adaptive inference with dynamic thresholds. |